S Hare or S Plit : Which Is More Efficient ?

In this paper, we investigate two different feature representations in convolutional neural networks (CNN): (a) Share all classes share the same feature space, and a fully connected layer is used to decode class activation, and (b) Split each class has its own feature space and a class is activated if its corresponding feature vector has a large norm. This is inspired by Capsules (Sabour et al. (2017)) which splits the feature space. We compare these two representations on a transformed MNIST dataset, which adds random scales and translations on the original digits. The experimental results show that Split has better performance when data is limited, while Share is better when data is big.